Skip to content

Conversation

amotl
Copy link
Member

@amotl amotl commented Dec 7, 2024

About

Adding an example using a convenient wrapper augmentation library around Apache Spark, that computes on the database instead of on a Spark cluster.

SQLFrame implements the PySpark DataFrame API in order to enable running transformation pipelines directly on database engines - no Spark clusters or dependencies required. It is based on SQLGlot and others.

References

@amotl amotl force-pushed the sqlframe branch 3 times, most recently from 851377f to 847f094 Compare December 7, 2024 16:15
SQLFrame [1] implements the PySpark [2] DataFrame API in order to enable
running transformation pipelines directly on database engines - no Spark
clusters or dependencies required.

[1] https://pypi.org/project/sqlframe/
[2] https://spark.apache.org/docs/latest/api/python/
With an aliasing to `char`, the response is:

cr> select char as "char" from t01;
SQLParseException[line 1:16: no viable alternative at input 'select char as "char"']

It apparently works with all other symbol names, it just trips on
`char`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant